Patient Education Materials Assessment Tool Research Articles

Navigating Scar Care: An Evaluation of Scar Treatment Patient Education Materials.

While patient education materials (PEMs) across various specialties have been reported as being too difficult to read, the quality and understandability of PEMs related to scar management have not been assessed. In this study, we report the breadth of scar management interventions and readability of online PEMs authored by academic societies and university hospitals. Websites of academic medical societies and university hospitals with scar revision PEMs were assessed for relevance. PEM readability was assessed via Flesch Reading Ease, Flesch-Kincaid Grade Level, and Gunning-Fox Index scores. Understandability and actionability were evaluated using the Patient Education Material Assessment Tool (PEMAT). A total of 26 scar revision PEMs met the inclusion criteria. The most commonly mentioned scar management interventions were scar revision surgery (73%) and laser scar revision (70%), with minimal emphasis on non-invasive methods like scar massage or sun protection. Readability analysis yielded a mean Flesch reading level of 8.8. Overall PEMAT understandability of online scar treatment PEMs was moderate, with a median of 76.0% (IQR 71.5 - 80.5%). PEMs from all specialties and institution types were lacking in actionability, with median actionability of 40.8% (IQR 38.1-60.0%). Online scar revision PEMs included a wide breadth of scar management interventions, however the least costly interventions of sun protection and scar massage were not commonly included. PEMs for scar management could be improved by simplifying language, including visual aids, and including checklists or specific steps patients can take to take action on scar management interventions.

Using Large Language Models to Generate Educational Materials on Childhood Glaucoma

PurposeTo evaluate the quality, readability, and accuracy of large language model (LLM) generated patient education materials (PEMs) on childhood glaucoma, and their ability to improve existing online information's readability. DesignCross-sectional comparative study. MethodsWe evaluated responses of ChatGPT-3.5, ChatGPT-4, and Bard to three separate prompts requesting they write PEMs on “childhood glaucoma.” Prompt A required PEMs be “easily understandable by the average American.” Prompt B required PEMs be written “at a 6th-grade level using Simple Measure of Gobbledygook (SMOG) readability formula.” We then compared responses’ quality (DISCERN questionnaire, Patient Education Materials Assessment Tool (PEMAT)), readability (SMOG, Flesch–Kincaid Grading Level (FKGL)), and accuracy (Likert Misinformation scale). To assess the improvement of readability for existing online information, Prompt C requested LLM rewrite 20 resources from a Google search of keyword “childhood glaucoma” to the American Medical Association-recommended “6th-grade level.” Rewrites were compared on key metrics such as readability, complex words (≥3 syllables), and sentence count. ResultsAll 3 LLM generated PEMs that were of high quality, understandability, and accuracy (DISCERN≥4, ≥70% PEMAT understandability, Misinformation score=1). Prompt B responses were more readable than Prompt A responses for all 3 LLM (p≤0.001). ChatGPT-4 generated the most readable PEMs compared to ChatGPT-3.5 and Bard (p≤0.001). Although Prompt C responses showed consistent reduction of mean SMOG and FKGL scores, only ChatGPT-4 achieved the specified 6th-grade reading level (4.8 ± 0.8 and 3.7 ± 1.9, respectively). ConclusionLLMs can serve as strong supplementary tools in generating high quality, accurate, and novel PEMs, and improving the readability of existing PEMs on childhood glaucoma.

Patient Education Materials for Immobilisation Masks in Radiation Therapy for Adult Head and Neck Cancer Patients: A Scoping Review.

Immobilisation masks (IMs) are used for people with head and neck cancer (HNC) undergoing radiation therapy (RT) treatment to ensure accuracy and reproducibility between treatments. Claustrophobia-related mask anxiety in HNC patients is common and can compromise treatment due to patient distress. This scoping review aimed to describe the content of publicly available Patient Education Materials (PEMs) for people with HNC undergoing RT. Three search engines (Bing, Yahoo, and Google) were systematically searched using standard terms. PEMs in audio-visual or written formats were eligible for inclusion if the target readership was adults with HNC and included content on IMs for RT. Content was appraised using the Patient Education Materials Assessment Tool for Printable and Audio-Visual Materials to assess understandability and actionability. In total, 304 PEMs were identified of which 20 met the inclusion criteria. Sixteen PEMs were webpages, three were PDF format, and one was a standalone video. The understandability and actionability of PEMs ranged between 47 to 100% and 0 to 80%, respectively. PEMs authored by Foundations/Organisations scored higher in understandability (80-100%) and were more likely to discuss mask anxiety coping strategies. In comparison, News sites and IM manufacturers published PEMs with the lowest understandability scores (20-80%). The significant variations in the quality of IM PEMs identified suggest that some sources may be more effective at informing patients about IMs. Although multiple aspects of the PEMs were consistent across the reviewed materials, many PEMs lacked information, and a stronger focus on understandability and actionability is required.

Evaluating Readability, Understandability, and Actionability of Online Printable Patient Education Materials for Cholesterol Management: A Systematic Review.

Dyslipidemia management is a cornerstone in cardiovascular disease prevention and relies heavily on patient adherence to lifestyle modifications and medications. Numerous cholesterol patient education materials are available online, but it remains unclear whether these resources are suitable for the majority of North American adults given the prevalence of low health literacy. This review aimed to (1) identify printable cholesterol patient education materials through an online search, and (2) evaluate the readability, understandability, and actionability of each resource to determine its utility in practice. We searched the MEDLINE database for peer-reviewed educational materials and the websites of Canadian and American national health organizations for gray literature. Readability was measured using the Flesch-Kincaid Grade Level, and scores between fifth- and sixth-grade reading levels were considered adequate. Understandability and actionability were scored using the Patient Education Materials Assessment Tool and categorized as superior (>80%), adequate (50%-70%), or inadequate (<50%). Our search yielded 91 results that were screened for eligibility. Among the 22 educational materials included in the study, 15 were identified through MEDLINE, and 7 were from websites. The readability across all materials averaged an 11th-grade reading level (Flesch-Kincaid Grade Level=11.9±2.59). The mean±SD understandability and actionability scores were 82.8±6.58% and 40.9±28.60%, respectively. The readability of online cholesterol patient education materials consistently exceeds the health literacy level of the average North American adult. Many resources also inadequately describe action items for individuals to self-manage their cholesterol, representing an implementation gap in cardiovascular disease prevention.

Scrolling for Answers About Breast Implant Illness: Application of Validated Tools to Assess the Quality of Content on YouTube and TikTok

Introduction: Popular video-sharing platforms YouTube and TikTok offer a plethora of information on the topic of breast implant illness (BII). As a largely patient-driven phenomenon, it is important to understand the influence of social media on patient knowledge regarding BII. This study sought to evaluate the quality, reliability, visibility, and popularity of YouTube and TikTok videos about BII. Methods: Two validated tools for health information, DISCERN and the Patient Education Materials Assessment Tool (PEMAT), were utilized to evaluate the quality of information regarding the topic of BII on YouTube and TikTok. High DISCERN score indicates content of superior quality and reliability, while elevated PEMAT scores signify content that is easily understandable and actionable for viewers. The search phrase “breast implant illness” was used to screen videos, which were sorted based on relevance and view count. The first 100 videos that fulfilled inclusion criteria were independently graded by three reviewers. Results: TikTok videos of longer duration, a higher number of shares, and in the patient education category were all significantly associated with a higher total DISCERN score ( P &lt; 0.05). YouTube videos that included a provider or a patient were significantly more likely to have a higher total DISCERN and PEMAT understandability score (all P &lt; 0.05). Discussion of physician education, operation details, and patient experience was significantly associated with higher total DISCERN and PEMAT understandability scores (all P &lt; 0.05). Conclusions: Total DISCERN and PEMAT scores for videos regarding BII on two popular social media platforms are low. Video length, patient experience categorization, and the presence of a provider are worth considering when developing high-quality online content for breast reconstruction and augmentation patients.

Use of standardized patients in patient education practices of senior nursing students: A mixed-methods study

BackgroundThere is a need for innovative teaching practices in nursing education due to many factors, such as global changes, the rapid development of technology, the increasing number of students, and the recent pandemic. ObjectivesThis research was conducted using standardized patients to evaluate the attitudes and skills of senior nursing students toward patient education practices following the implementation of a patient education training program. DesignMixed-methods design. SettingsThe study was conducted at a nursing faculty. ParticipantsThe sample of the study consisted of 47 senior nursing students. InterventionThe students participating in the study were given a four-hour patient education training that included the preparation of patient education, preparation of materials, and effective presentation. MethodsA descriptive information form prepared by the researchers, the Patient Education Implementation Scale (PEIS), the Turkish version of the Patient Education Materials Assessment Tool for Printable Materials (PEMATTR-P), and the presentation skill evaluation form (PSEF) were used to collect quantitative data. Semi-structured interview forms were utilized to collect qualitative data. SPSS for Windows v. 25.0 and MAXQDA20 were used for the data analyses. Results with a p value of <0.05 were considered statistically significant. ResultsThe post-test mean PEIS scores of the students increased in the total scale and in all subdimensions. A significant difference was found in the understandability and actionability of patient education materials evaluated with PEMATTR-P (p < 0.05). The mean PSEF score of the students was 85.14 ± 9.25 points. Within the scope of the research, two main themes, namely emotions and efficacy, were determined. ConclusionsThis study confirms that structured patient education training, including the use of standardized patients, is important for supporting and developing nursing senior students' attitudes and skills toward patient education.

Development and validation of the information-motivation-behavioural skills model-based human immunodeficiency virus education kit for adolescents in Malaysia

The growing Human Immunodeficiency Virus (HIV) incidences and insufficient HIV knowledge among Malaysian late adolescents necessitate immediate attention to HIV prevention via education. This study aims to develop and validate an Information-Motivation-Behavioural skills (IMB) model-based education kit for adolescents, PREM-Kit, to educate on HIV prevention among Malaysian late adolescents. Utilizing the Analysis, Design, Development, Implementation, and Evaluation model, we conducted the study in three phases: needs assessment, development of PREM-Kit, and validation of PREM-Kit by applying the IMB model to map the PREM-Kit’s contents. PREM-Kit, developed in Malay language, consisted of an infographic flip chart and videos. Five multi-disciplinary experts validated the PREM-Kit using the content validity index (CVI), and 13 end-users validated the PREM-Kit using the Malay version of the Patient Education Materials Assessment Tool for Printable and Audiovisual Materials. The infographic flip chart comprised three modules covering 15 topics, and an animated video accompanied each module. PREM-Kit achieved CVI scores of 1.0 and median scores of over 80% for understandability and actionability. Overall, the newly developed IMB model-based HIV education kit has good content validity, is simple to comprehend and apply, and is ready for testing its effectiveness in improving adolescents’ knowledge, attitudes, and practices for HIV prevention.

An Environmental Scan and Appraisal of Patient Online Resources for Managing Rheumatoid Arthritis Flares.

To conduct an environmental scan and appraisal of online patient resources to support rheumatoid arthritis (RA) flare self-management. We used the Google search engine (last search March 2023) using the terms "rheumatoid arthritis" and "flare management." Additional searches targeted major arthritis organizations, as well as regional, national, and international resources. Appraisal of the resources was conducted by 2 research team members and 1 patient partner to assess the understandability and actionability of the resource using the Patient Education Materials Assessment Tool (PEMAT). Resources rating ≥ 60% in both domains by either the research team or the patient partner were further considered for content review. During content review, resources were excluded if they contained product advertisements, inaccurate information, or use of noninclusive language. If content review criteria were met, resources were designated as "highly recommended" if both patient partners and researchers' PEMAT ratings were ≥ 60%. If PEMAT ratings were divergent and had a rating ≥ 60% from only 1 group of reviewers, the resource was designated "acceptable." We identified 44 resources; 12 were excluded as they did not pass the PEMAT assessment. Fourteen resources received ratings ≥ 60% on understandability and actionability from both researchers and patient partners; 10 of these were retained following content review as "highly recommended" flare resources. Of the 18 divergent PEMAT ratings, 8 resources were retained as "acceptable" following content review. There is high variability in the actionability and understandability of online RA flare materials; only 23% of resources were highly recommended by researchers and patient partners.

On the transcreation, format and actionability of healthcare translations

In public-health crises, members of multilingual communities must be able to access, understand, trust and act upon behaviour-change messaging. The role of translators is therefore critical, not only for the relaying of information but also in the transcreation of texts, understood as adaptation to suit the characteristics of an intended audience. Failure to use transcreation may produce messaging that is culturally inappropriate and thus ineffective. This study analyses healthcare resources created by governments in Australia with a view to identifying formatting and other visual features that would benefit from transcreation. A mixed-method approach combined numerical evaluation of four documents using the Patient Education Materials Assessment Tool (PEMAT) and a bottom-up thematic analysis of the way the same texts were discussed by 58 members of a broad range of ethnocultural and linguistic groups in Victoria, Australia. The findings point to a need to go beyond the linguistic aspects of the translation and take into account the discourse organisation, layout, images and cultural appropriateness of health messaging. The implications of applying the PEMAT criteria are not only that start texts will become more accessible and better able to facilitate understanding-based trust relations, but also that translators are well placed to participate in the transcreations that may be required in the various target languages.

Analysis of the quality information provided by “Dr. YouTube™” on premature ejaculation

ABSTRACT Background: YouTube™ is an important online resource to access health-related online information by the public worldwide. However, the quality of information available on it has not been adequately characterized. Aim: To assess the quality and reliability of information available on the treatment of premature ejaculation (PME) on YouTube™ in the Hindi and English language videos. Materials and Methods: A total of 151 (Hindi: 109, English: 42) YouTube videos were selected for assessment. The quality was evaluated using structured tools: Patient Education Materials Assessment Tool (PEMAT); and a 5-point modified DISCERN questionnaire (Range: 1-serious shortcomings; 5-minimal shortcomings). PEMAT assesses the understandability and actionability of video as separate percentages. Results: Three most common treatments suggested for PME were Kegel exercise (22.5%), start–stop technique (21.9%), and antidepressant medications (20.5%). Antidepressant medications, stop-squeeze techniques, and psychotherapy were more frequently suggested in English videos, whereas ayurvedic or herbal medicines were more frequently suggested in Hindi videos. About two-thirds of videos presented information in an easy-to-understand and actionable manner (PEMAT scores ≥70%). Only 6% of videos had a DISCERN score of ≥4, indicating good overall quality of information presented in them. Conclusion: People likely to encounter poor-quality information when seeking information for PME treatment on YouTube. A large number of videos suggested ineffective or unproven treatment strategies for PME. Healthcare professionals need to be mindful of this while counselling patients, and guide them regarding useful and reliable sources of health information available online.

Generative Artificial Intelligence to Transform Inpatient Discharge Summaries to Patient-Friendly Language and Format

By law, patients have immediate access to discharge notes in their medical records. Technical language and abbreviations make notes difficult to read and understand for a typical patient. Large language models (LLMs [eg, GPT-4]) have the potential to transform these notes into patient-friendly language and format. To determine whether an LLM can transform discharge summaries into a format that is more readable and understandable. This cross-sectional study evaluated a sample of the discharge summaries of adult patients discharged from the General Internal Medicine service at NYU (New York University) Langone Health from June 1 to 30, 2023. Patients discharged as deceased were excluded. All discharge summaries were processed by the LLM between July 26 and August 5, 2023. A secure Health Insurance Portability and Accountability Act-compliant platform, Microsoft Azure OpenAI, was used to transform these discharge summaries into a patient-friendly format between July 26 and August 5, 2023. Outcomes included readability as measured by Flesch-Kincaid Grade Level and understandability using Patient Education Materials Assessment Tool (PEMAT) scores. Readability and understandability of the original discharge summaries were compared with the transformed, patient-friendly discharge summaries created through the LLM. As balancing metrics, accuracy and completeness of the patient-friendly version were measured. Discharge summaries of 50 patients (31 female [62.0%] and 19 male [38.0%]) were included. The median patient age was 65.5 (IQR, 59.0-77.5) years. Mean (SD) Flesch-Kincaid Grade Level was significantly lower in the patient-friendly discharge summaries (6.2 [0.5] vs 11.0 [1.5]; P < .001). PEMAT understandability scores were significantly higher for patient-friendly discharge summaries (81% vs 13%; P < .001). Two physicians reviewed each patient-friendly discharge summary for accuracy on a 6-point scale, with 54 of 100 reviews (54.0%) giving the best possible rating of 6. Summaries were rated entirely complete in 56 reviews (56.0%). Eighteen reviews noted safety concerns, mostly involving omissions, but also several inaccurate statements (termed hallucinations). The findings of this cross-sectional study of 50 discharge summaries suggest that LLMs can be used to translate discharge summaries into patient-friendly language and formats that are significantly more readable and understandable than discharge summaries as they appear in electronic health records. However, implementation will require improvements in accuracy, completeness, and safety. Given the safety concerns, initial implementation will require physician review.

Assessing the Quality and Clarity of YouTube Videos as an Informational Source for Ankle Fractures: A Study of the Turkish Population.

Background and objective The significance of YouTube as a prominent source of healthcare-related information cannot be overstated. Its influence on patient education is substantial. However, despite its popularity, there has been little research into the quality and comprehensibility of videos related to ankle fractures, a condition with increasing incidence rates, particularly in the context of Turkey. To address this gap in the literature and the growing reliance on digital platforms for health information during the coronavirus disease 2019 (COVID-19) pandemic, this study aims to assess the quality, clarity, and pedagogical value of YouTube videos on ankle fractures for the Turkish population. Methodology A comprehensive examination of 150 YouTube videos concerning ankle fractures, employing exacting inclusion and exclusion criteria to identify 52 videos for in-depth analysis was undertaken. The investigation entailed an assessment of content quality, presentation style, and user interaction, utilizing the criteria established by the Journal of the American Medical Association (JAMA) and the Patient Education Materials Assessment Tool (PEMAT), as well as a novel measurement known as the Ankle Fracture Content Score (AFCS). A statistical analysis was executed to gauge the educational value and dependability of the information presented. Results The majority of ankle fracture management videos were found to be uploaded by medical professionals, covering various aspects such as rehabilitation and patient testimonials. However, the overall educational quality was suboptimal, with patient-shared videos lacking in depth and accuracy. Statistical analysis showed no significant differences in content quality across different uploader categories, except for notably lower educational quality in videos shared by patients. Conclusions The findings underscore a significant need for healthcare professionals and academic institutions in Turkey to produce and share high-quality, reliable, and comprehensible videos on ankle fractures. Leveraging YouTube's extensive reach can significantly improve health literacy among the Turkish public, ensuring access to accurate and trustworthy health information.

An effort to improve the collection of patient-generated data: readability and understandability of patient-reported outcomes measures in a survivorship cohort.

In this study, we evaluated readability and understandability of nine French-languagePatient-Reported Outcome Measures (PROMs) that are currently used in a contemporary longitudinalcohort of breast cancer survivors as part of an effort to improve equity in cancer care and research. Readability of PROMs was assessed using the Flesh Reading Ease Score (FRES), the Gunning's Fog Index (FOG), and the FRY graphics. Readability was considered ideal if mean score ≤ 6th-grade level and acceptable if between 6th and 8th grade. Understandability was evaluated using the Patient Education Materials Assessment Tool and defined as ideal if PEMAT ≥ 80%. The Evaluative Linguistic Framework for Questionnaires (ELF-Q) provided additional qualitative elements to assess understandability. Plain-language best practice was met if both readability and understandability were ideal. None of the 9 PROMs evaluated had ideal readability scores and only 1 had an acceptable score. Understandability ranged from 55% to 91%, and only 3 PROMs had ideal scores. ELF-Q identified points for improvement in several understandability dimensions of the PROMs. None of the instruments met the definition of plain-language best practice. None of the studied PROMs met the standards of readability and understandability. Future development and translation of PROMs should follow comprehensive linguistic and cultural frameworks to ensure plain-language standards and enhance equitable patient-centered care and research.

A health literacy analysis of online patient-directed educational materials about mycobacterium avium complex

IntroductionMycobacterium avium complex (MAC) is a complex lung infection requiring multi-disciplinary approach and management. Due to limited clinician-patient interactions, clinicians may refer patients to online resources to learn about the diagnosis, prognosis, and treatment of MAC. The American Medical Association (AMA) recommends educational materials be written at a sixth-grade reading level and the National Institutes of Health (NIH) recommends that patient education materials be written at an eighth-grade reading level; however, several evaluations found these materials inaccessible due to high literacy levels. To date, there has never been a health literacy assessment of MAC patient education materials. The study aims to assess the health literacy of online patient education materials about MAC. MethodsThe patient education materials were evaluated for readability, actionability, understandability and clarity. Readability was assessed through the Flesch-Kincaid Grade Level Scale (FkGL), SMOG Index, Coleman Liau Index (CLI), Gunning Fog Index (GFI), and Automated Readability Index (ARI). Actionability and understandability was evaluated using the Patient Education Materials Assessment Tool (PEMAT). The Centers for Disease Control (CDC) Clear Communication Index (CCI) was used to assess clarity. ResultsTen patient education resources were evaluated: CDC, Cleveland Clinic, Mayo Clinic, JAMA, American Thoracic Society (ATS), National Jewish Health, UpToDate, CHEST, WebMD, and Medline. The mean readability scores were as follows: FkGL (9.76), SMOG index (9.82), CLI (13.54), GFI (11.66), ARI (9.15). Four patient education materials were written at a sixth-grade reading level and eight patient education materials were written at an eighth-grade reading level. The majority of the materials received a passing score for understandability but failed to achieve a passing score for actionability. Cleveland Clinic, JAMA, and ATS all received a passing clarity score, indicating that they are easy to read. No patient education materials were available on UpToDate. ConclusionMost patient education materials scored poorly for actionability and clarity while scoring highly for readability and understandability. This study should serve as a guide for clinicians interested in offering online education materials to their patients. Increasing access to readable MAC educational materials should be a priority for those working at the intersection of public health, clinical care, and communications.

A Social Media Quality Review of Popular Sinusitis Videos on TikTok.

Social media may inform health care decisions among younger patient populations. TikTok is a social media platform that allows users to post short-form videos. This study aimed to assess the quality of sinusitis-related videos on TikTok. We searched TikTok on January 29, 2023, for sinusitis-related hashtags: #sinusitis, #sinus, #sinusinfection. Internet. The number of views/shares per day, uploader type (nonmedical influencer, lay individual, and medical professional) content categories (medical advice, marketing, comedy, and lifestyle/acceptability), and content type (educational vsfactual) were collected. The Patient Education Materials Assessment Tool for Audiovisual Material and Journal of the American Medical Association criteria score was used to measure understandability, actionability, and reliability. The Global Quality Scale (GQS) was used to evaluate the quality of videos; the harm/benefit score was used to evaluate causative effects. Analyses were performed using analysis of variance (α = .05). There were 221 videos identified, which garnered over 300 million views and 1 million shares. Almost half of the videos were published by nonmedical influencers. When controlling for covariates, nonmedical influencers and lay uploaders were more likely to have harmful harm/benefit scores, less understandable videos, and lower GQS scores compared to medical professionals. Less than half of videos posted by nonmedical influencers categorized as educational were factual (46.7%); lay individuals and medical professionals had higher rates of factual educational content (79.9% and 83.7%, respectively). Most nonmedical influencer-posted TikTok videos about sinusitis are inaccurate, despite being portrayed as medical advice/educational. Rhinologists must find modern ways to disseminate true disease-related content via social media to combat medical misinformation.

Dietary Phosphate Educational Materials for Pediatric Chronic Kidney Disease: Are Confused Messages Reducing Their Impact?

ObjectiveThis study aimed to review the quality and content of phosphate educational materials used in pediatric chronic kidney disease. MethodsThe quality of text-based pediatric phosphate educational materials was assessed using validated instruments for health literacy demands (Suitability Assessment of Materials [SAM] and Patient Education Material Assessment Tool [PEMAT-P]) and readability (Flesch Reading Ease [FRE] and Flesch-Kincaid [FK] Grade Level). Codes were inductively derived to analyse format, appearance, target audience, resource type and content, aiming for an intercoder reliability (ICR) >80%. The content was compared to Paediatric Renal Nutrition Taskforce (PRNT) recommendations. ResultsSixty-five phosphate educational materials were obtained, 37 were pediatric-focused including 28 text-based. Thirty-two percent of text-based materials were directed at caregivers, 25% children and 43% unspecified. Most (75%) included a production date with 75% produced >2 years ago. The median FRE test-score was 68.2 (interquartile range [IQR] 61.1-75.3) and FK Grade Level 5.6 (IQR 4.5-7.7). Using SAM, 54% rated “superior” (≥70), 38% “adequate” (40-69) and 8% “not suitable” (≤39). Low scoring materials lacked a summary (12%), cover graphics (35%) or included irrelevant illustrations (50%). PEMAT-P scores were 70% (IQR 50-82) for understandability and 50% (IQR 33-67) for actionability. An ICR of 87% was achieved. Over half limited foods in agreement with PRNT (including 89% suggesting avoiding phosphate additives). Recommendations conflicting with PRNT included reducing legumes and wholegrains. Over a third contained inaccuracies and over two thirds included no practical advice. ConclusionsText-based pediatric phosphate educational materials are pitched at an appropriate level for caregivers, but this may be too high for children under 10 years. The inclusion of relevant illustrations may improve this. Three quarters of materials scored low for actionability. The advice does not always align with the PRNT, which (together with the inaccuracies reported) could result in conflicting messages to patients and their families.

Evaluation of ChatGPT for Pelvic Floor Surgery Counseling.

Large language models are artificial intelligence applications that can comprehend and produce human-like text and language. ChatGPT is one such model. Recent advances have increased interest in the utility of large language models in medicine. Urogynecology counseling is complex and time-consuming. Therefore, we evaluated ChatGPT as a potential adjunct for patient counseling. Our primary objective was to compare the accuracy and completeness of ChatGPT responses to information in standard patient counseling leaflets regarding common urogynecological procedures. Seven urogynecologists compared the accuracy and completeness of ChatGPT responses to standard patient leaflets using 5-point Likert scales with a score of 3 being "equally accurate" and "equally complete," and a score of 5 being "much more accurate" and much more complete, respectively. This was repeated 3 months later to evaluate the consistency of ChatGPT. Additional analysis of the understandability and actionability was completed by 2 authors using the Patient Education Materials Assessment Tool. Analysis was primarily descriptive. First and second ChatGPT queries were compared with the Wilcoxon signed rank test. The median (interquartile range) accuracy was 3 (2-3) and completeness 3 (2-4) for the first ChatGPT query and 3 (3-3) and 4 (3-4), respectively, for the second query. Accuracy and completeness were significantly higher in the second query (P < 0.01). Understandability and actionability of ChatGPT responses were lower than the standard leaflets. ChatGPT is similarly accurate and complete when compared with standard patient information leaflets for common urogynecological procedures. Large language models may be a helpful adjunct to direct patient-provider counseling. Further research to determine the efficacy and patient satisfaction of ChatGPT for patient counseling is needed.

Artificial intelligence compared with human-derived patient educational materials on cirrhosis.

The study compared the readability, grade level, understandability, actionability, and accuracy of standard patient educational material against artificial intelligence chatbot-derived patient educational material regarding cirrhosis. An identical standardized phrase was used to generate patient educational materials on cirrhosis from 4 large language model-derived chatbots (ChatGPT, DocsGPT, Google Bard, and Bing Chat), and the outputs were compared against a pre-existing human-derived educational material (Epic). Objective scores for readability and grade level were determined using Flesch-Kincaid and Simple Measure of Gobbledygook scoring systems. 14 patients/caregivers and 8 transplant hepatologists were blinded and independently scored the materials on understandability and actionability and indicated whether they believed the material was human or artificial intelligence-generated. Understandability and actionability were determined using the Patient Education Materials Assessment Tool for Printable Materials. Transplant hepatologists also provided medical accuracy scores. Most educational materials scored similarly in readability and grade level but were above the desired sixth-grade reading level. All educational materials were deemed understandable by both groups, while only the human-derived educational material (Epic) was considered actionable by both groups. No significant difference in perceived actionability or understandability among the educational materials was identified. Both groups poorly identified which materials were human-derived versus artificial intelligence-derived. Chatbot-derived patient educational materials have comparable readability, grade level, understandability, and accuracy to human-derived materials. Readability, grade level, and actionability may be appropriate targets for improvement across educational materials on cirrhosis. Chatbot-derived patient educational materials show promise, and further studies should assess their usefulness in clinical practice.

ChatGPT vs. web search for patient questions: what does ChatGPT do better?

Chat generative pretrained transformer (ChatGPT) has the potential to significantly impact how patients acquire medical information online. Here, we characterize the readability and appropriateness of ChatGPT responses to a range of patient questions compared to results from traditional web searches. Patient questions related to the published Clinical Practice Guidelines by the American Academy of Otolaryngology-Head and Neck Surgery were sourced from existing online posts. Questions were categorized using a modified Rothwell classification system into (1) fact, (2) policy, and (3) diagnosis and recommendations. These were queried using ChatGPT and traditional web search. All results were evaluated on readability (Flesch Reading Ease and Flesch-Kinkaid Grade Level) and understandability (Patient Education Materials Assessment Tool). Accuracy was assessed by two blinded clinical evaluators using a three-point ordinal scale. 54 questions were organized into fact (37.0%), policy (37.0%), and diagnosis (25.8%). The average readability for ChatGPT responses was lower than traditional web search (FRE: 42.3 ± 13.1 vs. 55.6 ± 10.5, p < 0.001), while the PEMAT understandability was equivalent (93.8% vs. 93.5%, p = 0.17). ChatGPT scored higher than web search for questions the 'Diagnosis' category (p < 0.01); there was no difference in questions categorized as 'Fact' (p = 0.15) or 'Policy' (p = 0.22). Additional prompting improved ChatGPT response readability (FRE 55.6 ± 13.6, p < 0.01). ChatGPT outperforms web search in answering patient questions related to symptom-based diagnoses and is equivalent in providing medical facts and established policy. Appropriate prompting can further improve readability while maintaining accuracy. Further patient education is needed to relay the benefits and limitations of this technology as a source of medial information.

Gynecomastia Surgery Patient Education: An Information Quality Assessment of YouTube Videos.

YouTube is a platform for many topics, including plastic surgery. Previous studies have shown poor educational value in YouTube videos of plastic surgery procedures. The purpose of this study was to evaluate the quality and accuracy of YouTube videos concerning gynecomastia surgery (GS). The phrases "gynecomastia surgery" (GS) and "man boobs surgery" (MB) were queried on YouTube. The first 50 videos for each search term were examined. The videos were rated using our novel Gynecomastia Surgery Specific Score to measure gynecomastia-specific information, the Patient Education Materials Assessment Tool (PEMAT) to measure understandability and actionability, and the Global Quality Scale to measure general quality. The most common upload source was a board-certified plastic surgeon (35%), and content category was surgery techniques and consultations (51%). Average scores for the Global Quality Scale (x̄ = 2.25), Gynecomastia Surgery Specific Score (x̄ = 3.50), and PEMAT Actionability (x̄ = 44.8%) were low, whereas PEMAT Understandability (x̄ = 77.4%) was moderate to high. There was no difference in all scoring modalities between the GS and MB groups. Internationally uploaded MB videos tended to originate from Asian countries, whereas GS videos tended to originate from non-US Western countries. Patient uploaders had higher PEMAT Actionability scores than plastic surgeon uploaders. The quality and amount of gynecomastia-specific information in GS videos on YouTube are low and contain few practical, take-home points for patients. However, understandability is adequate. Plastic surgeons and professional societies should strive to create high-quality medical media on platforms such as YouTube.